最先进的语言模型从任何输入文本返回自然语言文本继续。这种生成相干文本扩展的能力意味着显着的复杂性,包括语法和语义的知识。在本文中,我们提出了一种数学框架,用于传递给定文本的扩展概率分布,例如由今天的大型语言模型学习的概率分布到包含语义信息的丰富类别。粗略地说,我们在文本上模拟概率分布作为富于单位间隔的类别。此类别的对象是语言中的表达,HOM对象是一个表达式是另一个表达式的概率。此类别是句法 - 它描述了与之相关的内容。然后,通过yoneda嵌入,我们将在此语法类别上传递给富集的单位间隔valued copreseaves。这类丰富的CopReseSeals是语义 - 我们找到了意义,逻辑运营,如蕴涵,以及更详细的语义概念的构建块。
translated by 谷歌翻译
这项工作起源于观察到,今天的最先进的统计语言模型不仅符合他们的性能,而且非常重要 - 因为它们完全从非结构化文本数据中的相关性建立。后一种观察会提示一个基本的问题在于本文的核心:非结构化文本数据中存在的数学结构是什么?我们提出了丰富的类别理论作为自然答案。我们展示了来自有限字母表的符号序列,例如在文本语料库中发现的那些,形成富含概率的类别。然后,我们解决了第二个基本问题:如何以保留分类结构的方式存储和建模此信息?我们通过从我们丰富的文本类别构建一个归力来回答这一点,以对特定的丰富的密度运营商类别。后者利用了积极的Semidefinite运算符上的Loewner订单,这可以进一步解释为一个有关的玩具例子。
translated by 谷歌翻译
A new method for solving the wave equation is presented, called the learned Born series (LBS), which is derived from a convergent Born Series but its components are found through training. The LBS is shown to be significantly more accurate than the convergent Born series for the same number of iterations, in the presence of high contrast scatterers, while maintaining a comparable computational complexity. The LBS is able to generate a reasonable prediction of the global pressure field with a small number of iterations, and the errors decrease with the number of learned iterations.
translated by 谷歌翻译
Actively monitoring machine learning models during production operations helps ensure prediction quality and detection and remediation of unexpected or undesired conditions. Monitoring models already deployed in big data environments brings the additional challenges of adding monitoring in parallel to the existing modelling workflow and controlling resource requirements. In this paper, we describe (1) a framework for monitoring machine learning models; and, (2) its implementation for a big data supply chain application. We use our implementation to study drift in model features, predictions, and performance on three real data sets. We compare hypothesis test and information theoretic approaches to drift detection in features and predictions using the Kolmogorov-Smirnov distance and Bhattacharyya coefficient. Results showed that model performance was stable over the evaluation period. Features and predictions showed statistically significant drifts; however, these drifts were not linked to changes in model performance during the time of our study.
translated by 谷歌翻译
Although prediction models for delirium, a commonly occurring condition during general hospitalization or post-surgery, have not gained huge popularity, their algorithmic bias evaluation is crucial due to the existing association between social determinants of health and delirium risk. In this context, using MIMIC-III and another academic hospital dataset, we present some initial experimental evidence showing how sociodemographic features such as sex and race can impact the model performance across subgroups. With this work, our intent is to initiate a discussion about the intersectionality effects of old age, race and socioeconomic factors on the early-stage detection and prevention of delirium using ML.
translated by 谷歌翻译
Graph neural networks (GNNs) have received great attention due to their success in various graph-related learning tasks. Several GNN frameworks have then been developed for fast and easy implementation of GNN models. Despite their popularity, they are not well documented, and their implementations and system performance have not been well understood. In particular, unlike the traditional GNNs that are trained based on the entire graph in a full-batch manner, recent GNNs have been developed with different graph sampling techniques for mini-batch training of GNNs on large graphs. While they improve the scalability, their training times still depend on the implementations in the frameworks as sampling and its associated operations can introduce non-negligible overhead and computational cost. In addition, it is unknown how much the frameworks are 'eco-friendly' from a green computing perspective. In this paper, we provide an in-depth study of two mainstream GNN frameworks along with three state-of-the-art GNNs to analyze their performance in terms of runtime and power/energy consumption. We conduct extensive benchmark experiments at several different levels and present detailed analysis results and observations, which could be helpful for further improvement and optimization.
translated by 谷歌翻译
Proteins play a central role in biology from immune recognition to brain activity. While major advances in machine learning have improved our ability to predict protein structure from sequence, determining protein function from structure remains a major challenge. Here, we introduce Holographic Convolutional Neural Network (H-CNN) for proteins, which is a physically motivated machine learning approach to model amino acid preferences in protein structures. H-CNN reflects physical interactions in a protein structure and recapitulates the functional information stored in evolutionary data. H-CNN accurately predicts the impact of mutations on protein function, including stability and binding of protein complexes. Our interpretable computational model for protein structure-function maps could guide design of novel proteins with desired function.
translated by 谷歌翻译
套件是指准备和分组必要的零件和工具(或“套件”)以在制造环境中组装。自动化此过程可简化人工工人的组装任务,并提高效率。现有的自动化套件系统遵守脚本指示和预定义的启发式方法。但是,鉴于零件和逻辑延迟的可用性差异,现有系统的僵化性可以限制装配线的整体效率。在本文中,我们提出了一个双重优化框架,以使机器人能够执行基于任务分割的零件选择,套件布置和交付计划,以及时提供定制的套件 - 即在需要时正确。我们通过人类主题研究(n = 18)评估了提出的方法,涉及基于研究的数据构建平板家具桌和购物流仿真。我们的结果表明,与使用由任务图本身定义的刚性任务分割边界定义的基线方法相比,与基线方法相比,与基线方法相比,即将到来的套件系统更有效,对上游商店流量延迟有弹性,并且比较更好地优选。单个套件,包括组装单个单元所需的所有零件。
translated by 谷歌翻译
图像引导放射疗法中的CBCT为患者的设置和计划评估提供了关键的解剖学信息。纵向CBCT图像登记可以量化分裂间的解剖变化。这项研究的目的是提出一个无监督的基于深度学习的CBCT-CBCT变形图像登记。提出的可变形注册工作流程包括训练和推理阶段,这些培训和推理阶段通过基于空间转换的网络(STN)共享相同的进率前路。 STN由全球生成对抗网络(Globalgan)和本地GAN(Localgan)组成,分别预测了粗略和细尺度运动。通过最小化图像相似性损失和可变形矢量场(DVF)正则化损失,而无需监督地面真实DVF的训练,对网络进行了训练。在推理阶段,训练有素的Localgan预测了局部DVF的斑块,并融合形成全图像DVF。随后将局部全图像DVF与Globalgan生成的DVF合并以获得最终的DVF。在实验中,使用来自20名腹部癌症患者的100个分数CBCT评估了该方法,并在保持测试中来自21名不同腹部癌症患者的队列中的105个分数CBCT。从定性上讲,注册结果显示了变形的CBCT图像与目标CBCT图像之间的对齐。定量地,在基准标记和手动确定的地标计算的平均目标注册误差(TRE)为1.91+-1.11 mm。变形CBCT和目标CBCT之间的平均平均绝对误差(MAE),归一化的跨相关性(NCC)分别为33.42+-7.48 HU,0.94+-0.04。这种有希望的注册方法可以提供快速准确的纵向CBCT对准,以促进分流的解剖变化分析和预测。
translated by 谷歌翻译
我们介绍了贴片采样时间表(PSS)的概念,该概念在训练过程中每批次使用的视觉变压器(VIT)贴片的数量变化。由于对于大多数视觉目标(例如,分类),所有补丁都不同样重要,因此我们认为,不太重要的补丁可以用于较少的训练迭代中,从而导致较短的训练时间,对性能的影响最小。此外,我们观察到,使用PSS的训练可以使VIT在推理过程中对更宽的贴片采样范围更强。这允许在推理过程中进行吞吐量和准确性之间的细粒度,动态的权衡。我们使用PSSS在VIT上评估Imagenet的VIT,均通过从头开始训练并使用重建损耗函数进行了预训练。对于预训练的模型,与使用所有斑块相比,我们的分类准确性降低了0.26%(从25小时到17小时)降低了0.26%。代码,模型检查点和日志可在https://github.com/bradmcdanel/pss上找到。
translated by 谷歌翻译